MSL meeting
February 28, 2023
“A study is reproducible if there is a specific set of computational functions/analyses that exactly reproduce all of the numbers and data visualizations in a published paper from raw data. Reproducibility does not require independent data collection and instead uses the methods and data collected by the original investigator.” (Marwick, 2016, p. 4)
Important
Crisis on replication and transparency in empirical research
in Nature (Baker, 2016)
“Many Labs 2” Project Results in November 2018
Comply with the demands for transparency
To collaborate more easily and effectively
Spot mistakes
Encourage learning and exploration
To communicate research more clearly
Workflows that integrate analysis and reporting
Gain visibility (Piwowar, Day, & Fridsma, 2007)
To raise awareness of quality concerns
Article – is just a tip of the iceberg. Reproducible Research makes the whole workflow accessible.
3 kinds of reproducibility (Stodden, Leisch, & Peng, 2014):
Computational reproducibility: code, software, hardware and implementation details.
Empirical reproducibility: detailed information about non-computational empirical scientific experiments and observations. Basically the data.
Statistical reproducibility: detailed information is provided about the choice of statistical tests.
Music Research is interdisciplinary, and some disciplines are closer to demands of reproducibility than others
Music Information Retrieval (MIR) leads the way (soundsoftware.ac.uk)
Music Psychology follows the path (interest in replication, but very slow take up (study about this coming up)
Music Intervention research (has to register study protocols)
Computational music analysis/theory/ethnomusicology utilise corpus studies
Workflows that are reproducible and transparent
Note
Sharing can be done anonymously for review (e.g. https://anonymous.4open.science, drupal.org, OSF)
“Reproducibility is like brushing your teeth. It is good for you, but it takes time and effort. Once you learn it, it becomes a habit.” (Baker, 2016)
slides available at:
MSL